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(54) Visual axis detection 

(57) In order to determine the direction of the gaze of the user of an automatic focusing camera, the user's 
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signal is then used to determine the user's direction of gaze; a suitable autofocus sensor can then be selected. 
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VISUAL AXIS DETECTION APPARATUS 

BACKGROUND OF THE INVENTION 
Field of the Invention 

The present invention relates to a visual axis 
detection apparatus, and especially to a visual axis 
detection apparatus which detects an axis in an 
observation point direction of a viewer (photographer) 
or a so-called visual axis when the viewer observes 
an observation plane (imaging plate) on which an object 
image is formed by a photographing system in an optical 
system such as a camera , by utilizing a reflected image 
(eyeball image) formed when an eyeball of the viewer 
is illuminated with an infrared ray. 
Related Background Art 

Various visual axis detection apparatuses for 
detecting the visual axis to detect a position on a 
view plane which the viewer (examined person) views 
have been proposed. 

For example, in Japanese Laid-Open Patent 
Application No. 2-264632, an infrared light beam from 
a light source is projected to an anterior eye in an 
eye to be examined and an axis of vision (observation 
point) is determined by utilizing a cornea reflected 
image on the basis of a reflected light from a cornea 
and a focus-imaging point on a pupil. 

In a camera disclosed in Japanese Laid-Open 
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1 Patent Application No. 61-61135, a direction of 

metering by a focus detection apparatus is mechanically 
controlled on the basis of an output signal from a 
visual axis detection means to adjust a focal point 
5 state of a photographing system. 

Fig. 5 is a schematic view of a visual axis 
detection apparatus proposed in Japanese Laid-Open 
Patent Application No. 2-264632, Fig. 6 is an explana- 
tion view for an output signal from one line of an 

10 image sensor of Fig. 5, and Fig. 7 is a perspective 
view of a portion of a finder system when the visual 
axis detection apparatus of Fig. 5 is applied to a 
single eye reflex camera. 

Numeral 101 denotes an eyeball of an examined 

15 person (observer), numeral 1 denotes a cornea of the 
eyeball of the examined one, numeral 2 denotes a 
sclera, and numeral 3 denotes an iris. O' denotes 
a center of rotation of the eyeball 101, O denotes - 
a center of curvature of the cornea 1, a and b denote 

20 ends of the iris 3, and e and f denote positions where 
cornea reflected images are formed owing to light 
sources 4a and 4b to be described hereinafter. Numeral 
4a and 4b denote light sources which may be light 
emitting diodes or the like for emitting infrared rays 

2 5 which are unpleasant for the examined one. The ? light 
source 4a (4b) is arranged closer to a projection lens 
6a (6b) than to a focal plane of the projection lens 



6a (6b). The projection lenses 6a and 6b are applied 
for widely illuminating the cornea 1 defining a light 
beam from the light sources 4a and 4b as diverged light 
beam. 

The light source 4a lies on an optical axis 
of the projection lens 6a and the light source 4b lies 
on an optical axis of the projection lens 6b, and they 
are arranged symmetrically along a z-axis direction 
with respect to an optical axis aXj. 

Numeral 7 denotes a light receiving lens which 
forms the cornea reflected images e and f formed near 
the cornea 1 and the ends a and b of the iris 3 on 
an image sensor plane 9. Numeral 10 denotes an 
arithmetic means which calculates the visual axis of 
the examined one by using the output signal from the 
image sensor 9. aXj denotes an optical axis of the 
light receiving lens 7 and it matching to an X axis 
in Fig. 5. aX 2 denotes an optical axis of the eyeball 
which makes an angle 9 to the X axis. 

In this example, the infrared ray emitted from 
the light source 4a (4b) passes through the projection 
lens 6a (6b) and thereafter widely illuminates the 
cornea 1 of the eyeball 101 with diverging state. The 
infrared ray which passes through the cornea 1 illumi- 
nates the iris 3. - 

The cornea reflected images e and f based on 
the light beam reflected by the surface of the cornea 
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1 1 of the infrared rays for illuminating the eyeball 
are reformed at points e' and f» on the image sensor 
9 through the light receiving lens 7. In Figs. 5 and 
6, e' and f denote projection images of the cornea 

5 reflected image (virtual images) e and f formed by 
a set of light sources 4a and 4b. Centers of the 
projection images e' and V substantially match to 
the projection point on the image sensor 9 of the cornea 
reflected image formed when the illumination means 
10 is arranged on the optical axis aXj. 

The infrared ray which is diffusion-reflected 
by the surface of the iris 3 is directed to the image 
sensor 9 through the light receiving lens 7 to form 
the iris image. 

15 0n the other hand, the infrared ray transmitted 

through the pupil of the eyeball illuminates a retina 
has the wavelength of the infrared range and the 
illuminated area is an area of a low view cell density 
which is apart from a center area, so that the examined 

20 one cannot recognize the light sources 4a and 4b. 

An ordinate in Fig. 6 represents an output 
I along the z-axis direction of the image sensor 9. 
Since most of the infrared ray transmitted through 
the pupil are not reflected back, there is no difference 

2 5 of the output at the boundary between the pupil ? and 
the iris 3. As a result, the iris images a' and b' 
at the ends of the iris can be detected. 
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1 When an area sensor having a two-dimensional 

photo-sensor array is used as the image sensor 9 of 
Fig. 6, two-dimensional light distribution information 
of the reflected image (eyeball image) is obtained 
5 from the front eye as shown in Fig. 8. 

In Fig. 8, numeral 141 denotes a light receiving 
area of the image sensors, E' and F' denote cornea 
reflected images of the light sources 4a and 4b, A' 
denotes a boundary between the iris and the pupil, 
10 and G' denotes a boundary between the sclera 2 and 

the cornea 1. since the reflectivities of the sclera 
1 and the iris 3 are not substantially different from' 
each other in the infrared range, the boundary G' can 
not be clearly discriminated by a naked eye. J * denotes 
15 an image of a lower eyelid, K' denotes an image of 

an upper eyelid and L« denotes an image of eyelashes. 

In order to detect the direction of the visual 
axis from the eyeball image of the front eye, it has 
been known to calculate a relative relation between 
20 the cornea reflected images e* and F' (or an interme- 
diate image of 5- and F') and the position of the center 
of pupil. Various methods for determining the center 
of pupil have beer, known. For example, an output of 
one particular line of the image sensor is sampled 
2 5 to calculate a center point of the pupil edge positions 
a' and b' of Fig. 6. Alternatively, the output 
information of the area sensor may be used to sample 
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1 a number of pupil edge points and thereafter determine 
the center point by a least square approximation. 

An optical equipment having a finder system 
such as a still camera or a video camera is frequently 
5 used in out-of-door. When such an optical equipment 
is used in out-of-door, the eyeball of the photographer 
is illuminated by an external ray. Thus, an image 
forming light beam received by the image sensor 9 
includes not only the image of the front eye illuminated 
10 with the light sources 4a and 4b but also a complex 
image affected by disturbance by the external ray. 

The most problem external ray is a direct light 
incident on the front eye from the sun. An energy 
of the sunlight is very strong and includes a plenty 
15 of the same spectrum components as those of the 
spectrums emitted by the light sources 4a and 4b. 
Accordingly, it is difficult to fully eliminate the 
external ray by spectrum means such as a visible ray 
cut filter. 

20 When the front eye is illuminated by the 

sunlight, a variety of disturbances are generated in 
the image. When the amount of external ray is largely 
illuminated, the exLetnal ray component is stronger 
than infrared component. As a result, a pattern 

2 5 (eyeball image) cannot be substantially discriminated. 
When the external ray exists, a brightness in the pupil 
which should be at a lowest brightness level of 
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1 luminescence (between a' and b' in Fig. 6) becomes higher 
or declined so that the detection of the pupil edges 
and hence the decision of the center of the pupil 
cannot be correctly determined. 

5 When the neighborhood of the boundary of the 

sclera and the iris is strongly illuminated, a obscure 
edge which inherently seems unclear rises to the surface 
or becomes declined therein, so that the pupil edges 
are misdetected. When the eyelashes grow downward, - 
10 they are illuminated by the external ray, so that they 
may be misdetected as the pupil edge. Since the 
eyelashes extend out of the face in contrast to the 
eyeball, they are easily subject to the illumination 
by the external ray. 

15 Such a misdetection occurs not only for the 

pupil edge but also for the cornea reflected images 
e and f of the light sources 4a and 4b. When the ends 
of the eyelashes are directly illuminated by the* \ 
sunlight, they become strong brilliant points, which 

20 are misdetected as the cornea reflected images. When 
eyeglasses are put, dusts deposited on the eyeglasses 
may be highlighted. 

Besides the sunlight, a down light having high 
luminescence and various artificial light sources are 

25 also utilized as the external ray. When eyeglasses - 
are put, a distance between the eyepiece portion in 
the finder system and the eyeball generally becomes 
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1 apart, so that the external ray easily enters into 

the eye. Further, the reflection coming from the lens 
surfaces of the eyeglasses is adversely affected. 

When the visual axis is to be detected by using 

5 the image signal from the image sensor, an accumulation- 
type image sensor is frequently used in view of a 
requirement for the sensitivity. As a result, there 
has been a problem that a DC noise elimination by an 
AC coupling or a period detection system which is 
10 usually used in <* single sensor cell cannot be used. 

The present invention is concerned with 
providing a visual axis detection apparatus for detecting 

15 an eyeball image by using accumulation-type image pickup 
means which reduces an affect by a noise due to an 
external ray and detects the visual axis of the eyeball 
of the photographer (the examined person) who looks into 
a finder, by properly setting an accumulation method 

20 of the eyeball image (image information) by the image 
pickup means and a processing method of the image 
information based on the eyeball image from the image 
pickup means. 

In the visual axis detection apparatus of the 
2 5 present invention, the eyeball of the examined person 

is illuminated by a light beam coming from illumination 
means, an eyeball image based on a reflected light 
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1 from the eyeball is formed on a surface in accumulation- 
type image pickup means, an image signal from the image 
pickup means is stored in memory means, and a visual 
axis of the examined person is calculated by utilizing 
5 the image signal stored in the memory means • The image 
pickup means has first and second accumulation periods 
and the memory means stores the image signal of the 
eyeball generated in one of the two accumulation periods 
and the illumination means emits a light in one of - 

10 the two accumulation periods, A difference signal 
between the image signal from the image pickup means 
generated in the first accumulation period and the 
image signal generated in the second accumulation period 
is determined by differential signal generation means 

15 and the visual axis of the examined person is detected 
based on the signal from the differential signal 
generation means. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 shows a main schematic view of the 
embodiment 1; 

Fig. 2 shows a flowchart of the embodiment 1; 

Fig. 3 siiuws a main schematic view of an image 
sensor of the embodiment 2; 
25 Fi 3- 4 shows a flowchart of the embodiment 2; 

Fig. 5 shows a main schematic view of a 
conventional visual axis detection apparatus; 
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Fig. 6 shows an explanation view of output 
signal from the image sensor in Fig. 5; 

Fig. 7 shows a main schematic view when the 
visual axis detection apparatus is applied to a single 
reflex camera; 

Fig. 8 shows an explanation view of an eyeball 
image formed on an area sensor; 

Fig. 9 shows a view when the visual detection 
apparatus is mounted into a single reflex camera; and 

Fig. 10 shows a block diagram for explaining 
how the apparatus in Fig. 9 is controlled. 

DETAILED DESCRIPTION OF THE PREFERBEQ EMBODIMENTS 

Fig. 1 shows a schematic diagram of Embodiment 
1 of the present invention and Fig. 2 shows a flow 
chart for explaining the visual axis detection in the 
Embodiment 1. 

In the present embodiment, in contrast to the 
conventional visual detection apparatus of Fig. 5, 
a photo-electrically converted signal from the image 
sensor 9 which functions as the accumulation-type image 
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1 pickup means is processed by arithmetic means 101, 
and a RAM (memory) 21 for storing the data from the 
arithmetic means 101 is further provided. Specifically, 
the visual axis operation method is improved in the 

5 arithmetic means 101 by using the data stored in the 
RAM 21 to eliminate the adverse affect by the external 
ray. 

The elements of the present embodiment are 
now explained in sequence although it may be partially 

10 duplicate to the description for Fig. 5. 

In Fig. 1, numeral 45 denotes an eyeball of 
an examined one (viewer) , numeral 1 denotes a cornea 
of the eyeball of the examined one, numeral 2 denotes 
a sclera and numeral 3 denotes an iris. O' denotes 

15 a center of rotation of the eyeball 10 i, 0 denotes 

a center of curvature of the cornea 1, a. and b denote 
ends of the iris 3, and e and f denote positions where 
cornea reflected images are generated by light sources 
4a and 4b to be described hereinafter. Numerals 4a 

20 and 4b denote light sources which may be light emitting 
diodes for emitting infrared rays which are unpleasant 
by the examined one. The light source 4a (4b) is 
arranged closer to a projection lens 6a (6b) than to 
a focal plane of the projection lens 6a (6b). The 

25 projection lenses 6a and 6b convert the light beams 
from the light sources 4a and 4b to diverging lights 
to widely illuminate on a surface of the cornea 1. 
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The light source 4a lies on an optical axis 
of the projection lens 6a while the light source 4b 
lies on an optical axis of the projection lens 6b and 
they are arranged symmetrically along a z-axis relative 
to an optical axis aX r The light sources 4a and 4b 
and the projection lenses 6a and 6b form the illumination 
means. 

Numeral 7 denotes a light receiving lens which 
focuses the cornea reflected images e and f formed - 
in the vicinity of the cornea 1 and the ends a and 
b of the iris 3 onto the image sensor 9. The light 
receiving lens 7 and the image sensor 9 form one of the 
light receiving means which converts the light 
from the eye into an electrical signal. 

Numeral 101 denotes an arithmetic means which 
calculates the visual axis of the examined person by 
using the output signal from the image sensor 9, as 
will be described hereinafter. The basic detection 
method therefor is described in Japanese Laid-Open 
Patent Application No. 4-447127. Numeral 11 denotes 
a RAM which functions as the memory means which stores 
data calculated by the arithmetic means 101. aX, 

x 

denotes an optical axis of the light receiving lens 

7, which matches with an X-axis. aX 2 denotes an. optical 

axis of the eyeball which makes angle 8 to the x'axis. 

In the present embodiment, the infrared ray 
emitted from the light source 4a (4b) passes through 
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the projection lens 6a (6b) and thereafter diverges 
to widely illuminate the cornea 1 of the eyeball 45. 
The infrared ray transmitted through the cornea 1 
illuminates the iris 3. 

The cornea reflected images e and f based on 
the light beam reflected by the surface of the cornea 
1/ of the infrared rays illuminating the eyeball are 
reimaged onto the points e* and f* on the image sensor 
9 through the light receiving lens 7. In Figs. 1 and- 
6, e' and f denote projection images of the cornea 
reflected images (virtual images) e and f generated 
by the set of light sources 4a and 4b. A mid-point 
of the projection images e' and f substantially matches 
to the projection position of the cornea reflected 
image on the image sensor 9, which is generated when 
the illumination means is arranged on the optical axis 
ax 2 . 

The infrared ray which is diffusion-reflected 
by the surface of the iris 3 is introduced into the 
image sensor 9 through the light receiving lens 7 to 
form the iris ir.iage. 

On the other hand, the infrared ray transmitted 
through the pupil of the eyeball illuminates the retina 
and is absorbed thereby. However since the illuminated 
area has a low density of viewing cells which is ? apart 
from the center, the examined one cannot discriminate 
the light sources 4a and 4b. 



1 In Fig. 6, an ordinate represents an output 

I in the z-axis of the image sensor 9. Since most 
of the infrared rays transmitted through the pupil 
are not reflected back, there arises a difference in 
5 the outputs at the boundary between the pupil and the 
iris 3 and the iris images a f and b 1 of the iris edges 
are detected. 

In the present embodiment , the arithmetic means 
101 respectively detects coordinates (Za f , Zb' and _ 
10 Ze 1 , Zf 1 ) of peculiar points (a 1 , b' and e', f) on 

the eyeball on the image sensor 9 based on a flow chart 
of Fig. 2, and calculates a rotation angle 9 of the 
eyeball in accordance with a formula: 

8*OC-sin8 ^ (Za'+Zb/ )/2 - (Ze'+Zf')/2 
15 where 8 is a magnification factor of the light receiving 
optical system (^Lo/Iu). 

A vision angle of the eyeball is determined 
from the rotation angle 9 to determine of the subject. 

In the line of vision detector of the present 
20 invention, a distance Li between the position at which 
the cornea reflected image is generated and the light 
receiving lens 7 satisfies a relation of: 

(L x | Ze ? -Zf : | ) /L 0 Z 0 ^ CC/ (L!~L 2 +0C) 
where Z 0 is a spacing in the z-direction of the set 
25 of light sources 4a (4b) , and L 2 is a spacing in -the 
x direction between the light source 4a (4b) and the 
light receiving lens 7. 



15 



Thus, even if the distance between the line 
of vision detector and the eyeball changes, the distance 
L x may be calculated from the spacing Ize'-Zf'l of 
the two cornea reflected images. 

An operation of the visual axis detection 
apparatus is now explained with reference to the flow 
chart of Fig. 2. 

In a step 201, the detection operation of the 
visual axis starts. In a step 202, the light sources 
4a and 4b are turned on and at the substantial same time 
the process proceeds to a step 203 to start the first 
accumulation operation of the image sensor 9. The 
accumulation by the image sensor 9 may be controlled 
by comparing a real time accumulation amount motor 
signal with a predetermined reference, or by time 
control by hardware or software timer. 

The process proceeds to a step 204 at the 
substantial end time of the first accumulation of the 
image sensor to turn off the light sources 4a and 4b. 
The photo-electric conversion signals of the image 
sensor 9 are sequentially read through a loop of steps 
205-207 and the A/D converted electrical signals Pi 
of the cells are stored in the memory (RAM) 21. Where 
the image sensor 9 itself does not have a memory, func- 
tion, the image sensor 9 may senses the light and - 
error-move during reading the signals. Accordingly, 
the loop is designed to be completed in a sufficiently 
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1 short time in comparison with the accumulation time. 

Where the image sensor 9 includes an analog 
memory function, the signal charges may be temporarily 
shifted to the non-photosensitive memory and sequen- 
5 tially read into digital system at a low speed. The 
memory function of the present embodiment may be 
implemented as a CCD channel or a capacitor array. 

When the reading and storing of all of the 
required pixels are completed, the process proceeds- 
10 to a step 208 to conduct the second accumulation 
operation. The accumulation time of the second 
accumulation is substantially same as the accumulation 
time of the first accumulation done in the step 203. 

In the second accumulation operation, the light 
15 sources 4a and 4b are not turned on and the front eye 
image is sampled by only the external ray illumination 
to cancel the external ray components. In the present 
embodiment, the accumulation time may be reduced to 
one half and the read gain may be doubled in order 
20 to reduce the accumulation time while keeping the 
apparent signal quantity. 

When the second accumulation operation is 
finished, the photoelectric conversion signals of the 
image sensor are sequentially read through a loop of 
25 steps 209-211. ' - 

Then the arithmetic means 10 reads the signals 
Pi of the same pixels produced in the first accumulation, 
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calculates differences di between the signals Pi and 
the current signals Pi' and restores the result in 
the memory 21. 

In the present embodiment, the arithmetic means 
101 also includes a function of differential signal 
generation means for determining the differential signal 
Pi'- This operation is carried out for all the pixels 
so that the memory 21 has an image signal based on 
the eyeball image which substantially eliminates the 
contribution of the external ray due to the sunlight 
or the like. In the present embodiment, the direction 
of the visual axis is calculated in a step 212 based' 
on the above image signal so that the malfunction is 
prevented and the highly accurate detection of the 
visual axis is attained. 

Fig. 3 shows a schematic view of an image sensor 
(a sensor chip) 301 in Embodiment 2 of the present 
invention, and Fig. 4 shows a flow chart of the opera- 
tion of the present embodiment. Other elements of 
the present embodiments are substantially identical 
to those of the Embodiment 1. 

In Fig. 3, numeral 301 denotes a sensor chip 
having a well-known self-scanning system and a power 
supply and the like, and it is shown as a functional 
block in Fig. 3 for simplification. 

The sensor chip 301 is provided at the position 
of the image sensor 9 in Fig. 1. Numeral 302 denotes 



IS 

1 a photo-sensing block which is a CCD sensor having 

M x N areas. Fig. 3 shows a frame- trans fer-cype system 
which shares the photo-sensing unit with a transfer 
unit although the same function may be attained by 

5 an interline-type system. A masked column 303 is 
provided at a left end of the photo-sensitive area. 
It is a monitor pixel to detect a dark signal level. 
A transfer buffer 311, a first memory 321, a transfer 
buffer/horizontal read register 331, a second memory. 

0 unit 341, a transfer buffer/horizontal read register 
351, and a differential amplifier 361 are provided 
in sequence. The elements other than the photo-sensitive 
area are fully shielded from the light by an aluminum 
film or the like. 

5 An operation of the present embodiment is now 

explained with reference to a flow chart of Fig. 4. 

In a step 401, the visual axis detection 
operation starts. In a step 402, the light sources 
4a and 4b are turned on. At the substantially same 

0 time, the first accumulation operation of the image 

sensor 301 is started in a step 403, and after employing 
the accumulation to a predetermined monitor level or 
after a predetermined time, the accumulation is 
terminated. 

5 In a step 404, the light sources 4a and '4b 

are turned off, and in a step 405, the transfer 
operation is conducted. In the transfer operation, 



the signal charges accumulated in the photo-sensing 
unit 302 of the image sensor 301 are transferred to 
the memory unit 321 through the transfer buffer 311. 

The transfer method is well-known. In the 
illustrated frame -transfer-type system, the signal 
charges of the pixels are transferred downward one 
line per one clock. The entire image is transferred 
to the memory unit 321 by (N+l) clocks including those 
for the buffer. It is necessary that the time required 
for the transfer is sufficiently shorter than the 
accumulation time. In the present embodiment, the 
transfer rate of the CCD channel is determined by the 
hardware and it is sufficiently high, so that any problem 
does not arise. 

When the transfer is over, the process proceeds 
to a step 406 to conduct the second accumulation 
operation. Since the charges of the photo-sensing 
unit are evacuated by the previous transfer operation, 
a reset operation is not necessary but it may be 
conducted prior to the second accumulation if the 
circuit is designed to conduct the reset operation. 

In the second accumulation, the light sources 
4a and 4b are not turned on and the signal charges 
by only the external ray are accumulated. After 
completing the accumulation, the process proceed's to 
a step 407 to conduct the transfer. 

In the transfer operation, the signal charges 
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1 accumulated in the photo-sensing unit 302 are transfer- 
red to the memory unit 321 and at the same tine the 
signal charges stored in the memory 321 by the first 
accumulation are transferred to another memory unit 

5 341. Since they are simultaneously and parallelly 

proceeded, the signal charges by the two accumulations 
are not mixed and the transfers are completed by (N+l) 
clocks. Finally, the signal charges by the first 
accumulation are stored in the memory 341 and the signal 
10 charges by the second accumulation are stored in the 
memory 321. 

In the next sequence, the signal is read 
outwardly through a loop of steps 408-409. This 
sequence may be lower at speed than the transfer in 

15 the sensor chip owing to an external radial circuit 

but the signal may be read without regard to the sensing 
by the sensor because the signal charges have been 
transferred to the light-shielded memory unit. 

The signal charges stored in the memory units 

20 (321 and 341) are sequentially transferred, pixel by 
pixel, to the charge-voltage converters 332 and 352 
by the function of the horizontal line read registers 
331 and 351, and the signal voltages are applied to 
the input terminals of the differential amplifier 361. 

25 since the both horizontal registers 331 and 351 are 
operated by one clock simultaneously, the signals of 
the same pixel of the photo-sensing unit 302 produced 
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1 in the first and second accumulations are simultaneously 
applied to the positive and negative inputs of the 
differential amplifier 361, As a result, the image 
signal which the external ray components is subtracted 
5 therefrom appears at the output terminal 371. When 
it is done for all pixels, the process proceeds to 
a step 410 to calculate the visual axis. In the 
present embodiment, the affection caused by the 
external ray is eliminated in such a manner to attain 

10 a highly reliable signal. 

In the present embodiment, a capacitor array 
may be also used to eliminate the external ray in the 
sensor chip. An image sensor which temporarily stores 
the photo-excited image signal charges in the capacitor 

15 array through a current amplifier element such as a 
transistor and thereafter sequentially read them out 
has been known, and hence the elimination of the 
external ray which is functionally equivalent to the 
CCD arrangement described above may be attained* 

20 Only one set of the memory unit may be provided 

for the first accumulation signal and the second 
accumulation signal may be subtracted on the chip and 
the result is output:. Alternatively, it may be 
re-stored in the memory. The significance of the 

25 present invention is not limited by the specific. - 
details of the implementation. 

Fig. 9 shows a schematic diagram of an 
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embodiment in which the line of vision detector of 
the present invention is applied to a single reflex 
camera. 

In Fig. 9, numeral 31 denotes a photographing 
lens which comprises two lenses for the sake of 
convenience although it actually comprises more lenses. 
Numeral 32 denotes a main mirror which is skewed into 
a photographing path or retracted therefrom depending 
on a view state of an object by a finder system and 
a photographing state of an object image. Numeral 
33 denotes a sub-mirror which reflects a light beam 
transmitted through the main mirror 32 to a focal point 
detection apparatus 39 at a bottom of a camera body 
to be described later. 

Numeral 34 denotes a shutter and numeral 35 
denotes a photo-sensing member such as a silver salt 
film, CCD or MOS or the like solid state image pickup 
device, or an image pickup tube such as a videcon.- 

Numeral 36 denotes a focal point detection 
apparatus which comprises a field lens 36a arranged 
near a focusing plane, reflection mirrors 36b and 36c, 
a secondary image forming lens 36d, a diaphragm 36e 
and a line sensor 36f and the like including a plurality 
of CCD's. 

The focal point detection apparatus 36 in. the 
present embodiment uses a well-known phase difference 
system. Numeral 37 denotes an imaging plate arranged 
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on an anticipated focusing plane of the photographing 
lens 31, numeral 38 denotes a pentadaha prism for 
altering a finder optical path, and numerals 39 and 
40 denotes an image forming lens and a photometer ing 
sensor, respectively, for measuring an brightness of 
the object in the view field. The focusing lens 39 
is related in conjugate with the imaging plate 37 and 
the photometering sensor 40 through a reflection optical 
path in the pentadaha prism 38. 

An eyepiece lens 41 having an optical splitter 
41a is arranged behind an exit plane of the pentadaha 
prism 38 and it is used for the observation of the 
imaging plate 37 by the eye 45 of the photographer. 
The optical splitter 41a comprises a dichroic mirror 
which transmits a visible ray and reflects an infrared 
ray. 

Numeral 42 denotes a light receiving lens and 
numeral 44 denotes an image sensor having two-dimen- 
sionally arranged photo-electric element array such 
as CCD's as explained above, which is arranged in 
conjugate with ^he vicinity of the pupil of the eye 
45 of the photographer which is at a predetermined 
position with respect to the light receiving lens 4 2 
(corresponding to 9 in Fig. 1). Numeral 43 denotes 
an infrared ray emitting diode which functions a ? s the 
light source (corresponding to 4 in Fig. 1). 

Numeral 51 denotes a high intensity superimposing 
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1 LED which can be recognized even for a bright object. 
The emitted light is reflected by the main mirror 32 
through the projection lens 52 and vertically deflected 
by a fine prism array 37a arranged at a display area 

5 of the imaging plate 37 and reaches the eye 45 of the 
photographer through the penta prism 38 and the eyepiece 
lens 41. 

The fine prism arrays 37a are formed in frame 
shape at a plurality of points (metering points) 

0 corresponding to the focus detection area of the imaging 
plate 37, and they are illuminated by five corresponding 
superimposing LED's 51 (which are defined as LED-L1, 
LED-L2, LED-C, LED-R1 and LED-R2) . 

Numeral 53 denotes a view field mask which 

5 forms a finder view field and numeral 54 denotes an 

LCD in the finder for displaying photographing informa- 
tion outside of the finder view field. It is illuminated 
by an illumination LED (F-LED) 55. 

The light transmitted through the LCD 54 is 

0 introduced into the finder view field by a triangular 
prism 56 and it is displayed outside of the finder 
view field so that the photographer may recognize the 
photographing information. 

Numeral 61 denotes a diaphragm provided in 

5 the photographing lens 31, numeral 64 denotes an" . 
aperture driver including an aperture drive circuit 
70 to be described later, numeral 63 denotes a lens 
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drive motor, numeral 64 denotes a lens drive member 
including drive gears and the like, and numeral 65 
denotes a photo-coupler which detects the rotation 
of a pulse disk 66 coupled to the lens drive member 
64 and transmits it to the lens focal point adjusting 
circuit 70, which drives the lens drive motor based 
on the information from the photo-coupler 65 and the 
lens driving amount information from the camera to 
drive the photographing lens 31 into an in-focus 
position. Numeral 67 denotes a well-known mount contact 
point which is an interface to the camera and the lens. 

Fig. 10 shows an electric circuit built in 
the camera of the present embodiment, and the like 
elements to those of Fig. 9 are designated by the like 
numerals. 

Connected to a central processing unit (CPU) 
100 of a microcomputer built in the camera body are 
a visual axis detection circuit 101, a photometer '■ 
circuit 102, an automatic focal point detection circuit 
103, a signal input circuit 104, an LCD drive circuit 
105, an LED drive circuit 106, an I RED drive circuit 
107, a shutter control circuit 108, and a motor control 
circuit 109. Signals are exchanged with the focus 
drive circuit 70 and the aperture drive circuit 111 
arranged in the photographing lens through the mount 
contact point 67 shown in Fig. 9. 

An EEPROM 100a associated with the CPU 100 
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has a visual axis correction data memory function for 
correcting a individual differential error of the visual 
axis. 

As described above, the visual axis detection 
circuit 101 A/D-converts the output of the eyeball 
image from the image sensor (CCD-EYE) based on the 
difference between the output in the illuminated state 
and the output in the non-illuminated state and sends 
the image information to the CPU 100, which samples 
each of characteristic points of the eyeball image 
necessary for the detection of the visual axis in 
accordance with a predetermined algorithm and calculates 
the visual axis of the photographer based on the 
positions of the characteristic points. 

The photometer circuit 102 amplifies the output 
from the photometering sensor 40, logarithmically 
compresses it, A/D-converts it, and sends the output 
to the CPU 100 as the luminescence information of each 
sensor. In the present embodiment, the photometering 
circuit 40 has photo-diodes including SPC-L, SPC-C, 
SPC-R and SPC-A for photometering four areas. 

The line sensor 36 of Fig. 10 is a well-known 
CCD line sensor including five line sensors CCD-L2 . 
CCD-LI, CCD-C, CCD-R1 and CCD-R2 corresponding to the 
five metering points in the image. 3 

The automatic focus detection circuit (focal 
point detection circuit) 103 A/D-converts the voltage 



obtained from the line sensor 36f and sends it to the 
CPU 100. SW-1 denotes a switch which is turned on 
by a first stroke of a release button to start the 
photometering, the auto- focusing and the detection 
of the visual axis, SW-2 denotes a release switch which 
is turned on by a second stroke of the release button, 
SW-AEL denotes an AE lock switch which is turned on 
by depressing an AE lock button, and SW-DIAL1 and 
SW-DIAL2 denote dial switches provided in an electronic 
dial (not shown) which are connected to an up/down 
counter of the signal input circuit 104 to count on 
rotation clicks of the electronic dial. 

Numeral 105 denotes a well-known LCD drive 
circuit for driving the liquid crystal display element 
LCD. It can display the aperture value, the shutter 
speed and the preset photographing mode on the monitor 
LCD 72 and the LCD 54 in the finder simultaneously 
in accordance with the signal from the CPU 100. 'The 
LED drive circuit 106 turns on and off the illumination 
LED (F-LED) 55 and the superimposing LED 51. The IRED 
drive circuit 107 selectively turns on the infrared 
ray emitting diodes (IRED1-6) according to surrounding 
states. 

The shutter control circuit 108 controls a 
magnet MG-1, which, when actuated, drives a leading 
curtain, and a magnet MG-2 which drives a trailing 
curtain, to impart a predetermined amount of light 
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exposure to a photosensitive member. The motor control 
circuit 109 controls a motor Ml which winds up and 
rewinds a film, and a motor M2 which charges the main 
mirror 32 and the shutter 34. The shutter control 
circuit 108 and the motor control circuit 109 carry 
out a series of camera release sequence. 

In detecting the visual axis, the eyeball of 
the subject is illuminated by the light beam from the 
illumination means 43, the eyeball image is formed 
on the accumulation type image pickup means 4 4 based 
on the reflected light from the eyeball, the image 
signal from the image pickup means is stored in the ' 
memory means 21 (RAM) (Fig. 1), and the visual axis 
of the subject is calculated by using the image signal 
stored in the memory means. The image pickup means 
has first and second accumulation periods, the memory 
means stores the image signal of the eyeball image 
generated in one of the two accumulation periods, and 
the illumination means emits light in only one of the 
two accumulation periods. A difference signal between 
the image signal from the image pickup means generated 
in the first accumulation period and the image signal 
generated in the second accumulation period is 
determined by the differential signal generation means 
and the line of vision of the subject is detected by 
using the signal from the differential signal generation 
means. The high luminescence LED 51 illuminates the 
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1 point based on the calculated visual axis information 
and the focus is detected by the focal point detection 
circuit 103 for the object area corresponding to the 
illumination point and the photographing lens 31 is 
5 driven by the focal point adjusting circuit 70. 

In accordance with the present invention, when 
the eyeball image is to be detected by using the 
accumulation type image pickup means, the affect by 
the noise due to the external rays is reduced by 

10 properly installing the accumulation method of the 

eyeball image (image information) by the image pickup 
means and the processing method of the image information 
based on the eyeball image from the image pickup means 
so that the visual axis detection apparatus which can 

15 accurately detect the visual axis of the eyeball of 
the photographer (the examined person) who views the 
finder. 
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WHAT IS CLAIMED IS : 

1. A visual axis detection apparatus: 

conversion means for converting a light from 
an eye of an examined person to an electrical signal; 

illumination means for illuminating said 

eye; 

signal generation means for generating a 
signal relating to a difference between a first signal 
of said conversion means when illuminated by said 
illumination means and a second signal of said conver- 
sion means in the absence of illumination; and 

detection means for detecting a visual axis 
on the basis of the signal of said signal generation 



means . 



2. A visual axis detection apparatus according 
to Claim 1 wherein said illumination means includes 
memory means for storing the electrical signal of said 
.conversion means in the state where the eye is illumi- 
nated by said illumination means, and said signal 
generation means outputs a signal frora which is subtracted 
an electrical signal of said conversion means in the 
absence of illumination from the signal stored by said 
memory means. 

j 

3. A visual axis detection apparatus according 

to Claim 1 wherein said conversion means includes a frame 
transfer-type solid state image pickup means having a 
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first memory and a second memory, said first memory 
stores the electrical signal of said conversion means 
in the state where the eye is illuminated by said 
illumination means, and said second memory stores the 
electrical signal of said conversion means in the 
absence of illumination. 

4. A visual axis detection apparatus comprising: 
illumination means for illuminating an eyeball 
of an examined person; 

image pickup means for accumulating a light 
from the eyeball in a first period and a second period 
as electrical signals; 

control means for making said illumination 
means illuminate the eyeball for one of said first 
and second periods to generate a differential signal 
between an electrical signal of said image pickup means 
in said one period and an electrical signal of said 
image pickup means in the other period; and 

detection means for detecting a visual axis 
on the basis of said differential signal. 

5. Visual axis detection apparatus comprising means for 
illuminating a person's eyeball, and means for generating 
a signal representing the visual axis of the eyeball and* utilising 
a differential signal generated by light reflected from the "eyeball 
when illuminated by said illuminating means but compensated for by 
a measurement of light reflected from the eyeball when not illuminated 
by said illumination means. 



6. Visual axis detection apparatus 
substantially as hereinbefore described with reference 
to and as shown in any one of Figures 1 to A of the 
accompanying drawings. 
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